Skip to content

BUG: Fix query doesn't support equals or not equals with Python parse… #40563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

alexprincel
Copy link
Contributor

…r (#40436)

@alexprincel
Copy link
Contributor Author

Could some of the pandas maintainers let me know if the failed checks are normal ? They are unrelated to any changes I've made as far as I know.

@mzeitlin11
Copy link
Member

Could some of the pandas maintainers let me know if the failed checks are normal ? They are unrelated to any changes I've made as far as I know.

If you look at the failures in a run like https://github.com/pandas-dev/pandas/pull/40563/checks?check_run_id=2163292906, looks like they might be related?

@mzeitlin11 mzeitlin11 added Bug expressions pd.eval, query labels Mar 23, 2021
@jbrockmendel
Copy link
Member

They look related to me. Do these tests pass for you locally?

engine, parser = self.engine, self.parser
df = DataFrame({"x": np.random.choice(["A", "B", "C"], size=20)})
tm.assert_frame_equal(
df.query("x == 'A'", engine=engine, parser=parser), df[df.x == "A"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't write compound asserts, use
result=
expected=
tm.assert_frame_equal(result, expected)

@alexprincel
Copy link
Contributor Author

Could some of the pandas maintainers let me know if the failed checks are normal ? They are unrelated to any changes I've made as far as I know.

If you look at the failures in a run like https://github.com/pandas-dev/pandas/pull/40563/checks?check_run_id=2163292906, looks like they might be related?

I understand that my changes had a broader impact than I had anticipated. Some of these tests hard coded the == and != operators as NotImplemented, which I have undone and this is causing them to fail.

Will have to really dig around to understand if these changes I made are ok but simply failing tests which should be amended or if I may be introducing some bug(s). Will take much more time than anticipated.

@alexprincel
Copy link
Contributor Author

Ok, I dug further into this and I both understand more and less about the issue. I now understand that both the pandas and python visitors inherit from the same BaseExprVisitor albeit with different parameters for disallow and different preparser function.

However, I do not understand why In and NotIn were specifically disallowed for the python engine even though its treatment would be identical under both PythonExprVisitor and PandasExprVisitor. This appears quite intentional as removing In and NotIn from the PythonExprVisitor disallow list makes a few tests fail as they specifically expect queries using these bin ops to raise NotImplemented.

Could someone please educate me as to why this is desirable ?

@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

@jreback
Copy link
Contributor

jreback commented Oct 4, 2021

would take this if you can merge master and will look again

@mroeschke
Copy link
Member

Thanks for the PR, but it appears to have gone stale. If interested in continuing, please merge master and a maintainer can have a second look.

@mroeschke mroeschke closed this Nov 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug expressions pd.eval, query
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: DataFrame.query doesn't support equals or not equals for string columns with python parser
6 participants